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TO MEASURE SCHOOL QUALITY, GOALS MUST FIRST BE 
SPECIFIED, APPROPRIATE CRITERION MEASURES DETERMINED, AND 
RELEVANT STUDENT AND SCHOOL CHARACTERISTICS CONTROLLED. AT 
PRESENT, LITTLE ATTENTION IS FOCUSED ON THE REAL EDUCATIONAL 
OUTPUT OF THE SCHOOL, THE LEVEL OF EDUCATIONAL ATTAINMENT OF 
THE STUDENT. THE SCHOOL SYSTEM SHOULD BE EVALUATED BY ITS 
EFFICIENCY, MEASURED IN OUTPUT PER UNIT INPUT WITHIN 
CLASSIFICATIONS OF SCHOOLS GROUPED ACCORDING TO SIZE, FUNDS, 

type, and other meaningful divisions, for many educational 

GOALS, NO SATISFACTORY INSTRUMENTS HAVE YET BEEN DEVISED. 
RARELY HAVE EVALUATIONS OF SCHOOLS BEEN BASED ON SOUND 
RESEARCH DESIGN SO THAT SENSIBLE COMPARISONS BETWEEN SCHOOLS 
COULD BE MADE. ONE OF THE BETTER RESEARCH STUDIES HAS BEEN 
THE NEW YORK QUALITY MEASUREMENT PROJECT. IN THIS STUDY, A 
SCHOOL'S EFFECTIVENESS IN TEACHING BASIC SKILLS, MEASURED BY 
STANDARDIZED ACHIEVEMENT TESTS, WAS ASSESSED BY COMPARISON 
WITH OTHER SIMILAR SCHOOLS. THE PROJECT TALENT DATA BANK, THE 
IOWA EDUCATIONAL INFORMATION CENTER, THE CARNEGIE ASSESSMENT 
PROJECT, AND THE PENNSYLVANIA QUALITY EDUCATION PROJECT ARE 
ALSO MENTIONED. IF THE RELATIVE EFFECTS OF VARIOUS 
ALTERNATIVES FOR IMPROVING SCHOOL QUALITY COULD BE 
DETERMINED, THEN DECISIONS BASED ON THE MOST EFFICIENT 
ALTERNATIVES WOULD GIVE US THE MAXIMUM EDUCATIONAL OUTPUT FOR 

Dollar expenditure, this paper is national center for 
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Educational Quality: Definition and Measurementi^^ 



It Is unfortunate that In a society which talks so much about the 
need for the best possible education for Its youth we are forced by 
honesty to say that the evaluation of Its educational processes and " 
Institutions Is woefully behind the state-of-the-art. "Good schools," 
"excellence," "quality education" and other loose terms are bandied 
about without ‘definition. If we attempt to cut through the semantics 
we find that the terms are operationally defined by the criteria used 
In their evaluation. To some, therefore, a "good" school system is 
one which I *8 a highly paid staff and a low pupil-teacher ratio, to 
others It Is one which has an active PTA, or one which Is building new 
and beautiful structures, or one which Is expending' large. sums of money 
on modern technical learning aids, or one which Is Introducing innovations 
In techniques, materials, and curricula.. In short. It Is assumed that 
the school which does something more than other schools Is a better 

• 'I 

school. There Is an obvious hiatus here between school practices, ' 

policies, and facilities which are considered beneficial and their 
actual effect upon the students. Little focus Is put on the real 
educatlona output of the school, the level of educational attainment 
of the students; instead Inappropriate headcounts, such as the number of - 
students going onto prestige colleges, the number of Merit Scholars, the ■ 

1/ Adapted from, a paper presented at the meetings of the American 
Statistical Association, Philadelphia, September 8, 1965 



amount of scholarship funds awarded to the students, or in some cases, 
the average test scores of the school are used as criteria of school 
quality. And so the logical criterion measure of output, the amount 
of change or growth in the student himself brought about by the school 
is neglected. Under the egalitarian views of some of our educators, 
it is assumed that our schools do equally well for all students and 
develop them to the full extent of their potential- -whatever that 
misunderstood and troublesome cliche'' means. 

The school system should be evaluated, as any other operating 
system would be, by the efficiency of the system, measured in output per 
unit input within classifications of schools grouped according to their 
size, funds available, type, etc. In other words, compared to other 
similar schools how much "bang" is a school getting for its "buck." 

This is indeed a stark way of looking at the problem. We must discard 
descriptive characteristics of a school system as the valid criterion 
of what it accomplishes. Though chrome ornaments on an automobile add to 
its decorative appeal they in no way add to the performance of the vehicle 
itself; so too in education the performance of the system is the proper 
criterion, not its "good looks." But if indeed we wish to use the beauty 
of a school plant as a criterion measure let us do it, but let us not 
call the resulting scale "quality of education" or impute it to be a 

s 

measure of what goes on inside the school. 

It has been assumed tnat the more money a school spends the better 
it is. Only insofar as higher expenditures do indeed contribute to the 
educational attainment of the students, is per pupil expenditure a valid 
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criterion measure. In fact data has revealed that some schools with 
meager budgets are contributing more to the growth of the students 
than other less efficient, overrated schools with higher expenditures. 

The effects of other school characteristics, such as school policies 
and practices, upon student output must first be investigated before 
advocating school expenditure as valid measures of school quality. 

Schools have many educational goals and outcomes, such as the 
ability to think and evaluate constructively and creatively, the appre- 
ciation of our democratic heritage, the acquisition of good habits and 
attitudes, etc., and specific criteria are needed for evaluating the 
schools effectiveness in meeting each of the defined objectives. For 
many of such goals we have not yet devised satisiactory measuring instru- 
ments. However, one major goal of the schools which is conceded by 
everyone is the acquisition of basic skills in the use of words and 
numbers. Since standardized achievement tests currently in use by the 
schools measure the level of student achievement in the various basic 
skills, they can be used as a criterion measure of the schools effective 

ness in achieving this specific goal. 

As hot as the "pursuit of excellence" has been in the past decade, 

the quarry has eluded us because the searching parties have been few 
and ill-equipped. The members of educational- establishments have in 
the past been reluctant to evaluate themselves, today they are only 
slightly more receptive to the idea of finding out what is going on in 
the schools. They are concerned with the testing of students, and 
achievement test scores are, of course, proper criteria for the assess- 
ment of output of the schools; however, most schools unfortunately. 
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consider a perusal of the average test scores to constitute an evaluation 
of the school. Some school systems and States have even published 
local norm tables for various achievement tests with the bald statement 
that the norms were to be used to evaliate the quality of the schools 
without any recognition of the fact that the schools differed greatly 
in size, expenditure, staff, and facilities, and that they were educating ' 
students who differed greatly in their socio-economic status, family 
bacSj^round, ability, motivation, and past preparation. Rarely have 
evaluations of schools been based on a sound research design which con- 
trolled in some way the variations in the characteristics of the school 
and the student body so that sensible comparisons between schools could 
be made. It is evident that only when such controls are used should 
there be an attempt to proceed to investigate the effect of the treatment 
variables, those characteristics of the schools which are thought to en- 
hance educational attainment. 

Of all the studies of school quality done in the past few are worth 
reporting. One of the better research studies has been the New York 
Quality Measurement Project. This project, started back in 1957 by the 
Division of Research of the New York St'ate Education Department, selected 
103 schools throughout the State for longitudinal investigation. I.Q, 
data were collected and various grades were tested for three consecutive 
years with a battery of achievement tests. Some limited school data 
were also collected on school size, socio-economic level, type of com- 
munity, staff, and teacher training. Mort's ’’The Growing Edge,” an 
instrument for measuring the adaptability of a school, the degree to 
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which a school employs modern instructional techniques, was also used 
as a criterion measure. Schools were then grouped by socio-economic 

level and community type for comparisons and separate norm tables were 
derived for these groups. 

In this study a school's effectiveness in teaching the basic skills, 
measured by standardized achievement tests, was assessed by comparing 
its test results to those of other similar schools. After grouping the 
schools by communities which had similar characteristics there wei'c, of 
course, still great differences in the educational attainment within 
each group. Also there were great differences in the expenditures of 
the schools. The amount of money spent by a school is only a rough 
measure of its quality level, and so it is easy to find schools with 
high expenditures which do not produce students with high achievement 
and similarly, we can find efficient low expenditure schools which’ turn 
out students with high achievement. In the New York study it was found, 
on closer examination, that the relative effectiveness of a school system 
varies with the l.Q. score of uhe student, the socio-economic level of 
the student, the subject matter content, and sex of the student. It Is 
not simply a matter of one school being better than another, rather it 
is that one school does a better job with a particular type of student 
in a certain subject matter area. Schools have strengths and weaknesses 
in specific areas and are not simply universally good or bad. 

One wishes, however, that more comprehensive input data on the 
students and school characteristics had been available in the New York 
study so that the analysis could be further refined. For then students 




could be grouped, or controlled, by home background, attitudes toward 
school, etc. After having controlled student input we could go on to 
investigate and try to discover the important school characteristics 
which influence the attainment of the students. Just such compre- 
hensive dita is available in the Project TALENT Data Bank from their 
large 1960 study. Also, the Iowa Educational Information Center is 
now starting to collect interlocked comprehensive data on schools, 
including home- background data, test scores, teacher data, and 
school data, which could be used in studies of school quality. The 
recent Educational Opportunities Survey has, of course, now become 
the basic source of comprehensive data for use in exploring the mea- 
surement of educational quality and in developing a model of educa- 
tional attainment. 

Two major studies of school quality underway at the present are 
worth mentioning. One is the Carnegie Assessment Project which is now 
in the stage of developing testing instriunents to be used in a large 
nationwide study to measure the outcomes of the schools. Another 
study is the Pennsylvania Quality Education Project which is now 
nearing completion. This is an exploratory attempt to develop criteria 
of school quality and to assess measures and indices of these criteria. 

Cost and quality are often discussed together. As I mentioned 
earlier, high expenditure does not insure quality in a school. It is 
evidently the judicious use of funds on factors contributing to the 
educational attainment of the student which is of import. It would be 
of great value to the Office of Education if we could determine the 



relative effects of various possible alternatives for improving school 
quality. (And here we intend to use the concept of school efficiency 
as a measure of school quality, ) For then we could make our decisions 
on the basis of the most efficient alternative which would give us 
the maximum educational output for a given resource or dollar ex- 
penditure. 

In summary, the path ahead in the measurement of school quality 
leads to the very simple realization that in order to do this properly 
we must first specify our goals and determine the appropriate criterion 
measures, then we must control on relevent student and school 
characteristics. Because of the v;ide organizational variations in 
the schools throughout the nation it would be advantageous to group 
the schools by geographic region, type of school, school size, tax 
base, or any other meaningful classification. Also, because of the 
differential effects of school training the pupils need also be classi- 
fied by sex, home background, attitudes, etc. By using such a pro- 
cedure we can reasonably expect to improve our assessment of the 



schools. 



